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Abstract — Fully autonomous robots will often need to open 
doors and traverse doorways in order to freely operate within 
human environments, and assistive robots that open doors on 
command would potentially benefit the motor impaired. In 
spite of these opportunities, autonomous manipulation of doors 
remains a challenging problem after more than a decade of 
research. Until recently, published research has focused on one 
or two aspects of door opening, and included results from only 
a small number of tests on a single door. Within this paper we 
present a set of behaviors that enable a mobile manipulator to 
reliably open a variety of doors and traverse doorways using 
force-sensing fingers and a laser range finder. 

With this system, a user only needs to briefly illuminate a 
door handle using a green laser pointer, after which the robot 
autonomously locates the door handle, finds the manipulable end 
of the door handle, twists the door handle, and pushes the door 
open while traversing the doorway. The behaviors use sensory 
feedback to continuously monitor task-relevant aspects of the 
world and respond to common forms of variation in the task, 
such as whether the door is locked or unlocked, is blocked or 
unblocked, opens to the right or left, or has a handle that twists 
down clockwise or counterclockwise. 

We tested the robot in 30 trials with 6 different doors from 
an initial position over 1.6 meters away from the door handle. 
For the 24 trials with unlocked doors, the robot succeeded at 
the entire task in 21 trials (87.5% success rate). In the 6 trials 
with locked doors, the robot successfully detected that the door 
was locked in all 6 trials (100.0% success rate). For all 30 trials, 
the robot stopped in a safe manner without requiring human 
intervention after detecting failure or success at the task. 

We conclude with a discussion of how this work relates 
to several broader issues for intelligent manipulation within 
human environments, including the use of 3D locations to select 
behaviors, the generality of serialized sub-tasks, task-relevant 
features, active perception, force sensing, and methods for scaling 
systems to handle more tasks of greater complexity. 

I. Introduction 

This paper makes two main contributions. First, it addresses 
the complete door opening task: the human illuminates a door 
handle using a green laser pointer and the robot moves up 
to the door handle, aligns itself with the door, haptically 
localizes the handle, detects the moveable tip, twists the 
handle, pushes the door, estimates the extent of the doorway 
and then passes through the doorway while opening the door 
to provide clearance. 

Second, the robot uses sensory feedback to continuously 
monitor task-relevant aspects of the world and respond to 
common forms of variation in the task. The robot adjusts its 




Fig. 1. Four of the six doors that the robot successfully opened and traversed. 
The first row shows the robot twisting the door handles and the second row 
shows the robot after it has traversed the doorway. 




Fig. 2. Force sensing fingers 



actions in order operate doors and handles with varied dimen- 
sions and directions of operation. At the same time, it ignores 
many irrelevant features about the door, such as its color or 
the details of the handle's geometry, which enables the robot 
to open a variety of doors without additional programming 
or learning, see Figure 1. The robot constantly evaluates its 
progress towards completion of the task, and gracefully and 
safely stops if it detects failure due to a locked door, a door 
that must be pulled, lost contact with the door handle, and 
other common impediments. 

To accomplish both of these goals we have decomposed the 
overall task of opening a door and traversing a doorway into 
several sub-tasks, and developed behaviors that can accomplish 
each of these sub-tasks in series. These behaviors rely on high- 
fidelity force sensing at the end-effector, which is performed 
by two custom fingers we have developed, each of which has 
a six-axis force/moment sensor at its base, see Figure 2. 

Within this paper, we first give a brief overview of related 
work. We then describe the robotic platform. Next, we describe 



the component behaviors and how they are combined into 
a complete control system. Then, we present quantitative, 
empirical results for the robot's performance in a variety of 
situations. Finally, we conclude with a discussion of how 
this work illustrates several broader themes for intelligent 
manipulation within human environments. 

A. Related Work 

For well over a decade, the operation of doors has served 
as a challenge problem for mobile manipulation within human 
environments [1, 2]. Until recently, published research has 
typically focused on one or two aspects of door opening in 
isolation, such as navigating to a door, locating or twisting a 
door handle, or navigating through an open doorway [3, 4, 5, 
6]. Presentations of more integrated systems have often lacked 
details or included results from only a small number of tests 
on a single door [7, 8, 9]. Due to a lack of empirical validation 
and the use of specialized geometric models, maps of the 
environment, and assumptions specific to particular doors, the 
generality and robustness of most previous methods is unclear 
[1, 2, 10, 11, 12, 13]. 

Rhee et al [8] present a robot which grasps a door handle 
using map matching, vision, and tactile sensing and then 
pulls open the door. The size of the door, and direction of 
opening are fixed and results are shown for only a single 
door. Petersson et al [14] present an admittance controller that 
allows the robot to estimate the radius and axis of rotation of 
the door. It assumes that the robot starts out with a firm grasp 
on the door handle and that the door can always be pushed 
open. Niemeyer et al [15], propose estimating the velocity of 
the end-effector (and also the object) and applying a force in 
the direction of this estimated velocity, which also requires a 
firm grasp on the door handle. 

Schmid et al [11], present a static manipulator for opening 
cabinet doors and drawers. It assumes that the handle does not 
need to be twisted. The 3D model of the handle is known and 
grasp analysis is performed prior to opening doors or drawers. 
Petrovskaya et al [12, 13] describe a method of determining 
the position and orientation of a door handle by estimating a 
Bayesian posterior based on force measurements. A 3D model 
of the door handle is assumed to be known in advance and the 
robot executes a pre-planned trajectory to twist the handle. Ott 
et al [9] use an impedance controller for flexible joint robots 
to make a mobile manipulator open a door. The paper presents 
results on one door only and assumes the direction of twisting 
of the door handle. Unlike previous approaches, the controller 
for pushing open the door does not require a firm grasp on 
the handle and is similar to the door pushing behavior that we 
present. 

Klingbeil et al [16] use a vision-based learning algorithm 
that can detect and locate door handles and elevator buttons 
and visually decide the direction in which door handles 
should be turned. Once the vision system has estimated these 
properties, a pre-planned trajectory is executed for twisting the 
handle. Due to lack of force feedback, it is unclear whether 
the robot can respond to mis-classifications (e.g. trying to twist 
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Fig. 3. The mobile manipulator, El-E (pronounced "Ellie"), used in this 
paper. 



the handle in the wrong direction) and unexpected situations 
such as locked doors or the door handle slipping out during 
twisting. 

Brooks et al [17] present Cardea, a mobile manipulator 
that visually detects doors while navigating through corridors 
and shoves open slightly ajar doors using impedance control. 
Like our work, Cardea uses behavior-based control. However, 
Cardea assumes the direction of opening of the door and does 
not operate door handles. The article only reports on a single 
trial with a single door. 

II. The Robot 

The robot, El-E (pronounced "Ellie"), with which we per- 
formed the work in this paper is a statically stable mobile 
manipulator, shown in Figure 3, that consists of a 5-DoF Neu- 
ronics Katana 6M arm, an ERRATIC mobile base by Videre 
Design, and a 1-DoF linear actuator we call the "Zenither" that 
can lift the arm and various sensors from ground level to 90cm 
above the ground [18]. The ERRATIC platform has differential 
drive steering with two powered wheels and one passive caster 
at the back. All computation is performed onboard with a Mac 
Mini running Ubuntu GNU/Linux. We have written most of 
our software in Python with occasional C++ and make use of a 
variety of open source packages including SciPy, Player/Stage 
and OpenCV. 

For this work, El-E uses three distinct types of sensors. 
First, El-E uses a laser pointer interface that consists of an 
omnidirectional camera with a narrow-band green filter and a 
pan/tilt stereo camera that is designed to detect a green laser 
spot and estimate its 3D location [19]. Second, El-E uses a 
laser range finder attached to the bottom of the aluminum 
carriage attached to the Zenither. When at its lowest height, 
the laser range finder scans across the floor. When lifted higher, 
it can scan across the surfaces of desks, tables and shelves. 



Third, El-E uses force sensing fingers that we designed and 
fabricated for this work. 

To enable the manipulator to sense the forces being applied 
at the end-effector we have replaced the Katana Sensor Fingers 
that we used in our prior work with our own custom fingers. 
These are shown in Figure 2. Each finger is a strip of 
aluminum covered with elastic foam for passive compliance 
and is connected to the motor via a six-axis force/moment 
sensor from ATI Industrial Automation. This enables us to 
measure the resultant forces and moments being applied to 
each finger independently. The force/moment sensors are ATI 
Nano25 (with a calibration of SI- 125-3). 

For any configuration of the manipulator, we transform the 
forces and moments measured by the fingers into a coordinate 
frame which is fixed with respect to the mobile base and 
coincides with the base of the manipulator. Measuring the 
force vector as opposed to only the magnitude (e.g., by using 
simple pressure sensors) offers the advantage that we can make 
estimates about the contact geometry such as the angle of the 
door handle or the door. 

III. The Behaviors 
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Fig. 4. Block diagram which shows the different behaviors, how the robot 
transitions between them and the robot's interpretation of failure. 



In this section we describe the implementation of the 
behaviors that form the complete system (Figure 4). It's worth 
noting that some of these behaviors could be further described 
in terms of sub-behaviors. 

A. Locating the Door Handle 

The first step is to orient the mobile base nearly perpendic- 
ular to the door and get the manipulator in contact with the 
door handle. Orienting the mobile base perpendicular to the 
door is required so that the laser range finder can be used to 
estimate the extent of the doorway after the robot successfully 
twists the door handle. 

The user shines a laser pointer at the door handle and this 
gives the robot an estimate of the 3D location. The robot then 
uses its laser range finder to estimate its orientation relative 
to the door and moves such that it faces the door handle, is 



perpendicular to the door and at a distance of 0.7m away. The 
robot estimates the orientation of the door by splitting the laser 
scan into two sub-scans (left and right of the robot) from 2° to 
20° and -2° to -20°. It then fits a line using least squares to 
each of these sub- scans and chooses the better fit (line with 
lower residual error) as the estimate of the orientation of the 
door. 

The Zenither then raises the manipulator to 15cm above 
the estimated 3D location of the laser point. The mobile base 
moves forward until the force sensitive end-effector detects 
contact with the door. To compensate for errors, such as human 
error, error in the estimation of the 3D location of the laser 
point, and error in the odometry, the robot haptically searches 
for the door handle over the surface of the door around the 
3D location. It scans the area by moving its end effector 
horizontally across the door in increments of 5cm. After each 
horizontal motion, it uses the Zenither to go down by 30cm or 
until contact is detected. If the fingers detect contact with any 
part of the door handle (a force in the vertical direction) then 
the search is terminated. If the door handle is not detected up 
to 20cm on either side of the initial contact with the door, the 
robot is unsuccessful in finding the door handle and stops. 

B. Deciding if the Door is Locked 




Fig. 5. Finding the tip of the door handle that can be moved down or declaring 
the handle to be locked. Left image: Rigid tip, Right image: moveable tip. 

The robot's model of a door handle is based on its relevant 
features for manipulation which are its two tips, one of which 
is rigid and the other which can be moved, if the door is 
unlocked. If the robot twists the door handle at the tip, it 
maximizes the moment arm and minimizes the force it must 
apply to twist the handle. Also, by sensing whether the tips 
can be moved or not, the robot can determine whether the 
door is locked. To twist the door handle, the manipulator first 
uses force sensing to estimate the tips of the door handle - 
It searches in steps of 1cm along the line parallel to the floor 
and the surface of the door, until it either overshoots the door 
handle (no force in the vertical direction) or the end-effector 
comes in contact with the walls on the side of the door (forces 
in the horizontal plane). 

After finding the tips of the door handle, the robot tries to 
move the tips down and uses a force threshold to determine 
whether the tip is moveable or not (Figure 5). If both the tips 
are rigid, the robot declares the door handle to be locked. 
If both the tips move down, the robot aborts because the 
dynamics of the object do not match the robot's model of a 
door handle. If one tip is moveable and the other is rigid, the 
manipulator moves above the moveable tip and starts twisting 
the door handle. 



C. Twisting the Door Handle 



E. Pushing Open the Door and Traversing the Doorway 




Fig. 6. Twisting the door handle using force-feedback 

Figure 6 shows the robot twisting a door handle in the 
counter-clockwise direction. The robot assumes neither the 
twisting direction nor the radius of the door handle. It uses 
the direction of the contact forces between the finger and the 
door handle as an estimate of the direction perpendicular to 
the door handle. It then moves the end-effector along this 
direction through a distance of 2cm before re-estimating the 
angle of the door handle. The robot declares success if the 
force applied by the end-effector exceeds 20N. This is close to 
the maximum force that the Katana can apply. If the magnitude 
of the measured force goes below 2N, the end effector is 
assumed to have lost contact with the door handle. 

D. Deciding if the Door can be Pushed 




Fig. 7. Left to right: Robot twists the door handle until the end. Robot 
pushes the door and estimates the doorway. Robot pushes the door so that the 
handle can be released. 

After twisting the door handle, the robot determines whether 
the door can be pushed open or not and estimates the location 
of the moveable end of the door (Figure 7). Keeping the door 
handle twisted, it tries to push the door by 5cm using only its 
arm. The robot stores a laser scan before and after this push. 
It takes the difference of these two laser scans to perceive 
the motion of the door. It performs connected components on 
the points from the original laser scan that moved more than 
3mm. The points within 4cm of each other are considered 
part of the same component. The robot assumes that the largest 
connected component is the door. It then estimates the location 
of the moveable end of the door by looking at which end of 
this connected component has more motion between the two 
scans. 

If either the number of points in the largest connected 
component is less than a threshold (i.e. the manipulator did 
not push open the door) or the manipulator applies a force 
of greater than 20N in the direction perpendicular to the door 
(implying that the door is obstructed or must to be pulled open) 
the robot declares failure in pushing open the door. Otherwise, 
the robot moves the mobile base to push the door by 15cm 
so that keeping the door handle twisted is no longer required. 
The geometry and configuration of the manipulator ensures 
that 15cm will not result in the base colliding with the wall. 




Fig. 8. Left image: Positioning itself to prevent collision with the wall. 
Right two images: Traversing through the doorway while pushing the door 
open. 

If the door can be pushed open, the robot moves back and 
positions itself such that it does not collide with the wall close 
to the moveable end of the door. The assumption here is that 
the doorway is wide enough for the robot to pass through 
and thus it is sufficient to estimate only the moveable end 
of the door. To traverse through the doorway, the robot holds 
its arm out in front and moves forward until the end- effector 
detects contact with the door. It estimates the angle of the door 
using the component of the contact forces in the horizontal 
plane and uses the manipulator to push open the door to 
provide clearance (Figure 8). The contact forces are sufficient 
to decide in which direction the door needs to be pushed and 
no assumptions are needed about the location of the hinges of 
the door relative to the robot. If the force required to push the 
door out of the way is greater than 20N, the robot stops and 
declares that the door is obstructed or too heavy for it to push 
further. 

IV. Experiments 




Fig. 9. The starting position and orientation of the robot for four trials. 



Fig. 10. The door handles from the six doors used in the experiments. 

We carried out a total of 30 door opening trials using six 
different doors, as shown in Figure 9. For each trial, a person 
illuminated the door handle with a laser pointer and the robot's 
task was to open the door and pass through the doorway. 
There were five trials for each door. In one of these five 
trials the door handle was locked. In the remaining four trials, 
the door was unlocked and the robot started in two different 
orientations relative to the door and two different positions 
with respect to the door handle. The starting position of the 
robot was approximately 1.5m perpendicular to the door and 



TABLE I 
Results from 30 total trials with 5 trials for 6 different doors. 



Door # 


Door state 


Locate 
door handle 


Decide if 
locked 


Twist 
door handle 


Decide if 
door can be pushed 


Push and 
traverse 


1 


Unlocked 
Locked 


4/4 
1/1 


4/4 
1/1 


4/4 


4/4 


4/4 


2 


Unlocked 
Locked 


3/4 
1/1 


3/3 
1/1 


2/3 


3/3 


2/2 


3 


Unlocked 
Locked 


4/4 
1/1 


4/4 
1/1 


3/4 


4/4 


3/3 


4 


Unlocked 
Locked 


4/4 
1/1 


4/4 
1/1 


4/4 


4/4 


4/4 


5 


Unlocked 
Locked 


4/4 
1/1 


4/4 
1/1 


4/4 


4/4 


4/4 


6 


Unlocked 
Locked 


4/4 
1/1 


4/4 
1/1 


4/4 


4/4 


4/4 


Overall Success Rate 
Success Percentage 


29/30 
96.6% 


29/29 
100% 


21/23 
91.3% 


23/23 
100% 


21/21 
100% 



around 0.75m to the right or left of the door handle in the 
direction parallel to the door. The angle between the normal 
to the door and the robot was varied between -60° and 60°, 
see Figure 9. 

Table I shows the performance of the robot for each of the 
trials. 'Locate door handle' is deemed successful if the robot 
servos to the door and makes contact with the door handle 
using its end-effector. 'Decide if locked' requires the robot 
to correctly report whether the door is locked or not. 'Twist 
door handle' is successful if the door can be opened once the 
robot stops twisting the door handle. 'Decide if door can be 
pushed' reports if the robot correctly determined whether or 
not the door could be pushed open. Finally, 'Push and traverse' 
reports whether the robot successfully traversed the doorway 
while using the manipulator to push the door open. 

The only failure with 'Locate door handle' was when the 
robot made contact with the wall instead of the door and thus 
could not haptically find the door handle. One of the two 
failures with 'Twist door handle' occurred because the handle 
slipped out during twisting. The other failure occurred because 
the handle required a force greater than 20N to be completely 
twisted, which resulted in the robot transitioning to door 
pushing without twisting the handle completely. The current 
threshold of 20N is governed by the maximum payload of the 
Katana manipulator. We believe that a manipulator capable 
of exerting greater forces would be able to overcome this 
failure, while still being safe for humans and avoiding damage 
to the environment. In general, we found that some door 
handles require greater force to twist than others. Likewise, 
door handles can get partially stuck and require greater force 
to twist free. 

In all three failure cases, the robot detected failure and 
stopped in a safe manner without requiring human interven- 
tion. These three were the only failures in the 30 trials using 
6 different doors. In the trials where the door was unlocked, 
the robot successfully completed all the sub-tasks in 21 out of 
24 trials (87.5% success). 



V. Discussion 

We conclude this paper with a discussion of broader issues 
related to intelligent manipulation within human environments. 

A. A 3D Location to Select and Influence Behavior 

We have shown that providing a coarse 3D position near the 
door handle enables the robot to haptically find the door and 
then search for the handle with respect to the surface of the 
door, which becomes a 2D search problem over a small area 
that can be performed robustly and efficiently. This indicates 
that fully autonomous behavior might be achieved with a high- 
level attention system that provides 3D locations to lower- 
level, task- specific behavior systems. From this perspective, 
in our work the human user plays the role of a high-level 
attention system that selects a task-relevant 3D location. We 
have previously shown the usefulness of this approach to select 
behaviors for object grasping and delivery. 

B. Task-relevant Features 

The presented behaviors illustrate the value of task-relevant 
features in enabling the robot to generalize by ignoring irrel- 
evant features unrelated to the task. In contrast to approaches 
that use explicit models of environments, doors, door handles, 
and door kinematics [3, 4, 5, 6, 11, 12, 13], our system focuses 
on task-relevant features, such as the approximate location 
of the door handle, the orientation of the door, the ends of 
the door handle, the manipulable end of the door handle, the 
magnitude and direction of the force vector, and the boundaries 
of the moving door. 

C. Active Perception and Force Sensing 

The robot uses force sensing coupled with behaviors that 
actively move the end-effector in several distinct ways. 

1) Detecting Task-Relevant Locations Through Contact: 
First, the robot uses force sensing to detect contact and in 
doing so implicitly estimates the position of the end-effector 
with respect to task-relevant features in the environment, such 
as the surface of the door, the surface of the handle, and the 
end of the handle. As we have previously noted, this implicit 



estimation is used to find task-relevant features instead of 
fitting a detailed geometric model. 

2) Estimating the contact geometry: Second, the high- 
fidelity 3 DoF force sensing provided by the fingers enables 
the robot to estimate key aspects of the contact geometry 
between the end- effector and the door or door handle. In 
particular, the resultant force vector gives an estimate of the 
direction in which the end-effector should move and apply 
force. If the finger is held in place in the middle of twisting 
the door handle, the spring-loaded door handle will push into 
the finger generating a resultant force vector that is parallel 
to the door handle's direction of motion. While pushing the 
door, the reaction forces enable the robot to estimate whether 
the door opens to its left or right. The robot can thus move the 
manipulator in small steps in the appropriate direction while 
traversing the doorway. 

3) Detecting Manipulability, Success, and Failure: Third, 
the behaviors use force sensing to detect when task-relevant 
elements of the environment can be manipulated by assessing 
whether they can be moved by applying a force with a 
magnitude below a predefined threshold. This is used to decide 
which, if either, of the two ends of the door handle can be 
twisted, when the door handle can no longer be twisted, and 
whether or not the door can be pushed open. In effect, this 
decision uses the maximum force that can be safely applied 
by the mobile manipulator to decide if something can be 
manipulated by the mobile manipulator. This is reasonable 
for this robot which has a low maximum force. However, 
for other robots additional criteria would need to be used 
including safety to the environment and safety to people in 
the environment. 

In turn, the robot uses the manipulability of task-relevant 
components of the environment to detect progress, termination 
conditions, success, and failure. For example, when the handle 
can no longer be twisted, the robot transitions to door pushing. 
By pushing on the door and detecting whether it moves or 
not, the robot determines whether door handle twisting was 
successful or not. Similarly, if the end-effector slips off of 
the handle, the magnitude of the force would drop below a 
threshold, indicating a loss of contact, and failure. 

D. Scaling Systems to Handle More Tasks of Greater Com- 
plexity (Scaling Systems Through Failure Detection) 

By decomposing the overall task into subtasks with asso- 
ciated behaviors that robustly estimate transitions, one can 
readily add more behaviors in a coherent way. For example, 
failure conditions could lead to additional behaviors, such as 
behaviors for pulling open the door if pushing fails, trying to 
turn the handle again if it slips out, or gracefully disengaging 
from an unexpected situation. Moreover, we have shown that 
a substantial part of door opening and doorway traversal 
naturally decomposes into a serial chain of behaviors with 
branches for failure. When considered in conjunction with 
our previous work on object fetching, placement, and delivery 
[18], this work indicates that many manipulation tasks in 
human environments may be amenable to serial chains of 



behaviors, and that the composition of modular behaviors may 
enable robots to handle a variety of tasks of greater complexity. 
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